Non-reversible Parallel Tempering for Deep Posterior Approximation

Parallel tempering (PT), also known as replica exchange, is the go-to workhorse for simulations of multi-modal distributions. The key to success PT adopt efficient swap schemes. popular deterministic even-odd (DEO) scheme exploits non-reversibility property and has successfully reduced communication cost from quadratic linear given sufficiently many chains. However, such an innovation largely disappears in big data due limited chains few bias-corrected swaps. To handle this issue, we generalize DEO promote propose a solutions tackle underlying bias caused by geometric stopping time. Notably, scenarios, obtain nearly based on optimal window size. In addition, stochastic gradient descent (SGD) with large constant learning rates exploration kernels. Such user-friendly nature enables us conduct approximation tasks complex posteriors without much tuning costs.